Changing the landscape of European School of Oncology–European Society for Medical Oncology masterclasses in clinical oncology during the COVID-19 pandemic

Aims: This study aimed to assess the participants' evaluation of the European School of Oncology–European Society for Medical Oncology virtual masterclasses in clinical oncology (MCOs) organized during the pandemic in 2021. Materials & methods: The participants answered an online evaluation questionnaire at the end of each MCO to evaluate the content and organization of the MCO. Results: The clinical session and case presentation scores ranged between 4.6 and 4.8 over 5. The participants strongly agreed that the MCOs offered updates to improve their knowledge and practice in 68–83% and 52–76%, respectively; 74–90% of the participants considered the quality of the meetings to be excellent. Conclusion: The participants were satisfied with the virtual MCOs during the COVID-19 pandemic. Virtual MCO may be an acceptable alternative educational modality in specific circumstances.

professionals [8]. However, cancer education of young fellows and oncologists seemed to be overlooked during the pandemic, with many cancer educational events being canceled, interrupted or delayed [9].
The European School of Oncology (ESO) and ESMO masterclasses in clinical oncology (MCOs) are full, 5-day, interactive educational courses designed to enhance participants' knowledge and skills in medical oncology [10]. During an MCO, the ESO and ESMO faculty experts present in-person lectures that cover the latest diagnostic and treatment updates in oncology. Since 2002, ESO and ESMO have organized in-person MCOs in five different geographic regions yearly. The authors have previously reported the positive impact of ESO-ESMO MCOs on selfprofessional development, as 95% of the participants in MCOs considered this meeting useful for their professional career [11]. Before the pandemic, young oncologists attended the MCOs in person and commonly reported a high level of satisfaction with the goal, content and organization of the meetings [12,13]. Due to travel restrictions related to the COVID-19 pandemic, ESO canceled the MCO in 2020 and transitioned to a virtual environment in 2021 via e-ESO, an ESO e-learning platform. The authors conducted a survey to assess the participants' evaluation of the five ESO-ESMO virtual MCOs organized during the pandemic and compared the participants' evaluation with the previous in-person MCOs conducted between 2002 and 2019.

Virtual MCO description
The virtual MCOs were teaching courses designed for medical and clinical oncologists to discuss patient management by cancer site and the latest updates. As in in-person meetings, eligible participants were selected on a competitive basis according to their curriculum vitae by a panel of ESO and ESMO experts. Participants and lecturers were emailed an access link prior to the scheduled lecture to provide instructions on navigating the online conferencing platform. The virtual MCO consisted of 13 hours of pre-recorded educational lectures and a 2-day live, online educational event that focused on specific topics of interest for oncologists. The pre-recorded lectures were made available to the participants 1 month before the live, online event. Participants attended these courses and answered three evaluation questions related to each session. During the live, online event, each lecture was followed by a question-and-answer (Q&A) session using some of the questions that were mentioned in the pre-recorded sessions. After each section, the panel of experts discussed the latest updates and solicited questions from the participants for approximately 10 min. Prior to the meeting, participants had to work in pairs to prepare clinical cases, which were presented and discussed with the faculty members during the live, online event [14]. ESO opted for this activity to increase networking among attendees, given that this opportunity is often missing from online educational activities. At the end of the MCO, participants were required to complete a self-assessment test [15].

Evaluation of the MCOs
Participants were asked about their experiences with the virtual MCO using an online evaluation questionnaire that was addressed by an email invitation. Only one reminder email was sent thereafter; the reminder was not personalized and encouraged participants to answer the survey by offering the possibility to join the College of the European School of Oncology [16]. Responses were anonymous and could not be linked to the participant's score on the final self-assessment test.
A questionnaire of five sections was developed by the ESO faculty at the inception of the in-person MCO and was approved by the ESO ethics board [15]. The first section consisted of four demographic questions. In the second section, three items evaluated whether the MCO improved the participants' knowledge of and skills in the best clinical practice and ranked the quality of the education offered during the virtual MCO. The third and fourth sections covered the content and organization of the MCO in four and three items, respectively. The last section evaluated the presentations separately according to the slides' reliability, English quality and general feedback. Responses were collected using Likert scales: strongly agree, agree, neutral, disagree and strongly disagree or excellent, good, fair, poor and very poor. At the end of each section, freehand text answers were allowed to suggest areas of improvement.
Regarding the evaluation scores of the clinical sessions and clinical case presentations, the overall scores ranged between 4.6 and 4.8 over 5 in the different geographic regions; the lowest score was in the Arab countries and southern Europe MCO ( Table 3). Most of the clinical sessions evaluated by the participants, as well as the clinical case presentations presented by the participants, were highly valued, with an overall scoring of 4.5-4.9 and 4.7-4.9, respectively.
The participants strongly agreed that the MCO offered updates to improve their knowledge in the field and practice in 68-83% and 52-76%, respectively; 74-90% of the participants considered the quality of the meeting to be excellent. The Arab countries and southern Europe MCO participants reported the lowest satisfaction score (Table 4). Among the participants in MCOs, the content of the sessions was well balanced and supported with evidence in 58-77% and the time of the discussion sessions was satisfying in 52-68%. In the evaluation of the event organization, 81-89% of the participants considered the organization and management of the meeting to be excellent, albeit a lower satisfaction with the platform, ranging between 68 and 84%.
The Supplementary Material details the answers provided in the freehand text items intended to suggest areas of improvement. Overall, participants were satisfied with the content and organization. The main suggested area of improvement was relevant to the time of interaction for discussion and questions between the faculty and the participants.

Discussion
The COVID-19 pandemic led to an unprecedented cancelation of the ESO-ESMO MCOs during 2020, rather than an abrupt transition to a virtual teaching environment to deliver didactic lectures during an exhausting period. Nevertheless, the authors primary objectives were in line with the Accreditation Council for Graduate Medical Education (ACGME) and considered that training and teaching programs must continue to provide adequate resources, training and supervision [17]. As previously mentioned, ESMO had already published a series of guidelines for the management of patients during the pandemic. Thereafter, ESO designed e-ESO, a dedicated platform suitable for the MCO lectures and interactions.
This survey aimed to assess the first virtual MCO experience for participants by assessing their satisfaction in regard to the goals, content and organization of the masterclasses. Overall, the results reflect that the majority of the participants were satisfied with the MCOs despite the transition from the traditional in-person to virtual lectures. Interestingly, the item relevant to discussion time seemed to score the lowest, which may reflect the challenging part of virtual meetings with large audiences. Indeed, many participants suggested increasing the duration of the interaction with the experts in order to improve the quality of the virtual MCOs. Moreover, participants commonly   Table 4. Proportions of participants who responded 'strongly agree' or 'excellent' in the general, content and organization evaluation sections of the survey.

Baltic and Eurasia Eastern European and Balkans
Did the e-course offer qualified updates? 83% 83% 68% 71% 77% Did the e-course improve your skills and your practice?
Give an overall rating for the quality of the e-course. discuss their queries with the experts after their presentations during in-person events, whereas this accessibility to the experts is not possible during a virtual MCO. The authors attempted to understand the participants' satisfaction according to their geographic region and detected that the Arab countries and southern Europe MCO had the lowest score for the majority of the items relevant to the goals, content and organization of the meeting. Notably, participants in the Arab countries and southern Europe MCO had the highest proportion of satisfaction with the time dedicated for discussions with the expert panel. This finding underlines a differential need for information among the geographic regions [18][19][20]. Previous publications have shown that virtual teaching has not been favored over the traditional in-person approach [21][22][23][24]. However, these findings cannot be extrapolated to the medical education during the COVID-19 pandemic period, as the workforce was directed toward patient care and teaching activities were suspended. A recent publication reporting on the online education of oncology fellows during the pandemic has shown that most participants felt comfortable with the transition to a fully virtual didactic curriculum [25]. The MCOs occurred during a period with a growing optimism regarding the effectiveness of the COVID-19 vaccine in preventing severe infection and a trend toward involving, at least partly, in-person meetings. Nevertheless, the authors' experience showed that the virtual ESO-ESMO MCOs achieved higher satisfaction scores in comparison with historical controls from the previous 17 in-person MCOs across the five geographic regions (Figure 1). The survey did not detail the parameters that would influence such improvement because MCOs were originally designed to favor networking and social engagement. The higher differential satisfaction improvements in the western Europe MCOs  followed by the Arab countries and southern Europe MCOs may be attributed to the COVID-19 pandemic surges and cost/convenience benefits, respectively. A satisfaction survey among radiation oncology trainees showed that the reduced cost and travel time were the most common reasons to favor virtual meetings, especially among those training in non-metropolitan centers [26]. Although the current study may be generalizable to the different geographic regions involved, several limitations should be acknowledged. The number of participants in the virtual MCOs was similar to that in the in-person MCOs; nevertheless, the sample size considered for this analysis was quite small. Unfortunately, the authors did not address the survey to the oncologists who had registered for the course but dropped out; however, the concordance of the dropout rate with those of previous years did not allude to particular issues. Finally, this survey may have favored participants with easy access to electronic tools. Participants were not asked specifically about the challenges of the participation modalities, but two issues can be raised. Many participants complained about the short duration of the interaction with the experts and showed lower satisfaction concerning this item of the survey. A minority of the participants found the e-ESO platform complex but quickly adapted to its tools.

Conclusion & future perspective
The COVID-19 pandemic has accelerated the embrace of modern technologies in academic interactions to maintain continuous medical education, especially during difficult times when expert advice is required. The results of this survey demonstrate that virtual MCOs can be amenable to incorporation into oncology education. This positive feedback from participants from five different geographic regions supports the acceptability of the virtual approach in delivering medical oncology updates. Furthermore, the results of the survey are instructive in directing ESO-ESMO faculty members to adapt the virtual sessions according to the participants' needs.

Introduction
• This study aimed to assess the participants' evaluation of the European School of Oncology-European Society for Medical Oncology virtual masterclasses in clinical oncology (MCOs) organized during the pandemic in 2021.

Methods
• Participants answered an online evaluation questionnaire at the end of each MCO to evaluate the content and organization of the MCO.

Results
• Participants were highly satisfied with virtual MCOs during the COVID-19 pandemic.
• The majority of the participants considered the quality of the meetings to be excellent.
• The Arab countries and southern Europe virtual MCOs had the lowest evaluation.
• The western Europe virtual MCOs had higher improvement in evaluation compared with the in-person events.
• Participants complained mainly of the duration of the interaction with the experts. Conclusion • Virtual MCOs may be an acceptable alternative educational modality in specific circumstances.

Supplementary data
To view the supplementary data that accompany this paper please visit the journal website at: www.futuremedicine.com/doi/